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Abstract 


In this paper I will provide an introduction to the theory of cellular 
automata and review a paper by Michele D’amico, Giovanni Manzini, 
and Luciano Margara [3], On computing the entropy of cellular au- 
tomata, in which the authors study the topological entropy of cellular 
automata. The main problems addressed are proving a closed form for 
the topological entropy of D-dimensional linear cellular automata over 
Zm for D = 1 and for D > 2 and showing how to efficiently compute 
the entropy of positively expansive cellular automata. I will cover the 
closed form for the entropy of linear CA. 


Figure 1: A “pulsar” in Conway’s Game of Life. 
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1 Introduction 


1.1 What are cellular automata? 


Cellular automata, abbreviated CA, are a class of discrete models often 
studied in computability theory, theoretical biology, and physics. A cellular 
automaton is a regular grid of cells with a finite number of states, for example 
black or white as in Figure 1. Then over a discrete number of time steps, t, 
the state of each cell evolves based on the states of the cells in a neighborhood 
of itself. The configuration after each time step is called a generation. 

'The concept was originally discover in the 1940s by Stanislaw Ulam and 
John von Neumann at Los Alamos National Laboratory. It was not until the 
1970s, though, that CA were studied in academia. It was the Game of Life, 
also known as Life, a cellular automaton devised by British mathematician 
John Conway in 1970. 


1.2  Conway's Game of Life 


Conway's Life is a zero-player game, meaning that its evolution is determined 
by its initial state with no further input. The game is played on an infinite 
grid of square cells where each cell has two states, alive or dead. Each cell 
interacts with the eight cells around it each time step to determine its new 
state. The rules are as follows: 


1. Any live cell with fewer than two live neighbors dies, as if caused by 
under-population. 


2. Any live cell with two or three live neighbors lives on to the next 
generation. 


3. Any live cell with more than three live neighbors dies, as if by over- 
crowding. 


4. Any dead cell with exactly three live neighbors becomes a live cell, as 
if by reproduction. 


'The game is particularly interesting because it is a universal Turing machine, 
that is, anything that can be computed can be computed within the Game 
of Life. It is this game that opened up the field of cellular automata. 


2 Applications 


The theory of cellular automata is being used to create ecological, biological, 
and physical models to great success. In this section I will briefly cover two 
different practical applications that arose from the study of CA. 


2.1 Forest Fires 


In [5] a model is presented for predicting the spreading of fire in homoge- 
neous and inhomogeneous forests. The model can easily incorporate weather 
conditions and land topography. Fires are a part of almost all natural ecosys- 
tems, and over the course of many centuries have exerted an exceptionally 
important influence on the condition of forests all over the planet. The en- 
vironmental effects of forest fires are huge and there is a constant demand 
for more effective tools to manage and fight these fires. The authors present 
the first cellular automata approach to modeling forest fires. They state the 
problem as follows: 


Given a scalar velocity field R(x,y) which is the distribution 
of the rates of fire spread at every point in a forest, the forest fire 
front at time tı, the wind direction and speed, and the height and 
shape of the land, determine the fire front at any time tz > ty. 


To create the model the authors partitioned the forest into a uniform grid of 
square cells. The state is then given by the ratio of burned out cell area to 
total cell area. So a cell state of 1 would mean that the area around the cell 
is completely burned out, while a state of 0 would be completely unburned. 
'Then each time step moves the fire a cell length divided by the speed of fire 
spreading in each direction. This creates a dynamic model for the spread of 
forest fires which were in good agreement with fire spreading in real forests. 
'This model served as the first basis for the development of algorithms that 
simulate real fires in real forests. 


2.2 Settlement Patterns 


In [4] the author applies a cellular automata algorithm to predict the devel- 
opment of a rural countryside near Toronto, Canada. The model was created 
similarly to the forest fire model in [5], the area was broken up into a grid 
and environmental factors such as major roads and land topography were 
used to create the different states that each cell could attain. Two scenarios 
were run: (1) a static set of rules; and (2) rules that changed as conditions 


or policies within the township changed. Both models were in agreement 
with each other and the actual development until the second scenario’s rules 
began to change with the township’s own policies. After that point both 
models still captured and represented selected aspects of the real system. 
Scenario 1 added new houses mainly on major roads and the density of clus- 
ters was quite similar to measured data. While scenario 2 failed to match 
densities, but more accurately showed the spatial distribution of the clusters. 


These are simply two of the early models that came out of the study of 
cellular automata. This paper will be focused on the results proven by M. 
D’amico et al. in [3]. The paper focuses on topological entropy, which is 
one of the most studied properties of dynamical systems. The topological 
entropy measures the uncertainty of the forward evolution of a dynamical 
system when a complete description of initial configurations is unknown. Be- 
cause cellular automata are deterministic dynamical systems, given a com- 
plete description of any configuration we may exactly determine the future 
configurations. Topological entropy gives a quantitative estimation of the 
uncertainty that is introduced when a complete configuration is not known. 


3 Definitions 


3.1 Cellular automata 


Let Z and N denote the set of integers and natural numbers, respectively. 
Let Zm, m > 2, denote the finite commutative ring of integers modulo m, 
that is, for m > 2 let, Zm = (0,1,...,m — 1}. We define the space of 
configurations, «P = (ele EOS Zm}, which consists of all functions 
from ZP into Zm. An element of (ond can be visualized as an infinite D- 
dimensional lattice in which each cell contains an element of Zm. 


Let s > 1. A neighborhood frame of size s is an ordered set of distinct 
vectors U1, ua, ..., us € ZP. Given f : Z$, — Zm, which we call a local 
rule, à D-dimensional cellular automata based on f is defined as the 
pair (D, F), where F : 6? > «P, is the global transition map defined 
as follows. 


[F(c)](v) = f(c(v + u1), ..., c(v + uj)), where c e G2, v e ZP. (1) 


That is, any cell v in the configuration F(c) is a function of the content of 
cells v + u1, v+ Ug, ..., V+ Us in the configuration c. The local rule f and 


the neighborhood frame completely determine F. 


F(c) will denote the result of the application of the map F to the con- 
figuration c, and c(V) will denote the value assumed by c in v. For n > 0, 
we recursively define F"(c) by F"(c) = F(F"-1(c)), where F°(c) = c. Let 
(€P, F) be a CA based on the local rule f. We say that f is permutative 
in the variable rj, —r < i € r, if and only if, no matter which values are 
given to the other 2r variables, the modification of the value of x; causes 
the modification of the output produced by f. We denote by f™ the local 
rule associated to F”. 


3.2 Linear CA over Zm 


In the case of a linear CA the set Zm has the usual sum and product 
operations that make it a commutative ring. We will denote [z],, to be 
the integer z taken modulo m. Linear CA have a local rule of the form 
f(zxwu...,.) = Doa Niti]m with A1,...,A.s € Zm. Hence, for a linear 
D-dimensional CA (1) becomes 


[F(c)](v) = 5 Ajc(v-c-uj)| , where ce EP, v e ZP. (2) 
i-l mn 
For linear 1-dimensional CA the local rule f can be written as f(x ,,...,2,) = 


[Dp aiz;]m where at least one between a_, and a, is nonzero. In this case 
(1) becomes 


[F(c)](i) = »» ajc(i--j)| ,wherece €l, ic Z. 
jo-r » 


3.3 Topological entropy of CA 


The topological properties of CA are usually defined with respect to the 
metric topology induced by the Tychonoff distance over the configuration 
space P. With this topology VP is a Cantor set, that is it is a compact, 
perfect and totally disconnected set, and every CA is a uniformly continuous 
map. The definition of topological entropy # of a continuous map F : X > 
X over a compact space X, denoted (X, F), is generally accepted as a 
measure of the complexity of the dynamics of F over X. It is defined in [1], 
but informally it is a measure of the uncertainty of the evolution of the CA 
given only partial information on the initial conditions. It is uncomputable 


for general CA, but in the linear case we will be able to give a closed form, 
which is the goal of the paper. The entropy of a 1-dimensional CA over € 
satisfies (F) < 2rlogm. 


3.4 Lyapunov exponents for CA 


The Lyapunov exponent of a dynamical system is a quantity that char- 
acterizes the rate of separation of infinitesimally close trajectories and is 
denoted by A. 

For every zr € EL and s < 0 we set 


Wi (x) = {y € €} : y(i) = z(i) forall i > s}, 


W; (x) = {y € €l: yli) 2 z(i) forall i < —s}. 


We have that W;* (x) C W$; (x) and W; (x) C Wi,(x). For every n > 0 
we define 


An (£) = min{s > 0 : F"(Wo (x)) C W$ (F”(2))}, 


À; (z) = min(s > 0: F”(W5 (z)) C W; (F^(a))). 


Simply put, Af (x) and A; (x) have a simple meaning. Wy (x) is the set of 
configurations which agree with zx in all cells with index i > 0. By comparing 
F" (z) with F” (Wi («)) the value A7 (x) measures how far differences in cells 
with index 4 < 0 can propagate to the right-hand side in n iterations of F. 
Similarly, A> (x) measures how far differences in cells with index i > 0 can 
propagate to the left-hand side in n iterations of F. We will now introduce 
an important 1-dimensional CA, the right shift map (@!,c) defined by 
[c(c))(i) = c(i — 1). Now we can introduce the following shift invariant 
quantities: 


A, (z) = max À; (of (£)),  Aj(z) = max Aj;(e(z)), (3) 
jeZ jeZ 
where c denotes the right shift map. Finally, the values A*(z) and A^ (x) 
defined by 
1 
+ ee . — + =e = " k 

A (x)= lim —Az(z), A (x)= lim A, (2) (4) 

are called, respectively, the right and left Lyapunov exponents of the CA 


F for the configuration x. If F is linear it is easy to see that these do not 
depend on z. 


3.5 Dynamical properties of CA 


For any discrete time dynamical system defined on a metric space, it is 
possible to define important properties which provide useful information on 
the long term behavior of the system. On a generic system (X, F), where 
F : X — X, where we assume that X is equipped with a distance d and 
that the map F is continuous on X according to the metric topology induced 
by d. For CA the Tychonoff distance satisfies this property. We denote by 
B(x, €) the open set (y € X : d(x,y) < €}. 


A discrete time dynamical system (X, F) is positively expansive if and 
only if there exists ô > 0 such that for every x,y € X, x Æ y there exists 
n > 0 such that d(F"(x), F"(y)) > ô. The value ô is called the expansivity 
constant. 

Intuitively a map is positively expansive if every pair of enough close 
points eventually separate by at least 6 under iteration of F. If a map is 
positively expansive, then, for all practical purposes, the dynamics of the 
map defies numerical approximation. Small errors in computation which are 
introduced by round-off become magnified upon iteration. 


A discrete time dynamical system (X, F) is sensitive to initial conditions 
if and only if there exists 6 > 0 such that for any x € X and for any e > 0, 
there exists y € A(x,e) and n > 0, such that d(F"(x), F"(y)) > ô. The 
value 6 is called the sensitivity constant. 

A map is sensitive to initial conditions, or simply sensitive, if there exist 
points arbitrarily close to x which eventually separate from x by at least ô 
under iteration of F. Note that not all points near x need eventually sep- 
arate from x under iteration, but there must be at least one such point in 
every neighborhood of x. 


A discrete time dynamical system (X, F) is equicontinuous at x € X 
if and only if for any ô > 0 there exists e > 0 such that for any y € A(z, c) 
and n > 0 we have d(F” (x), F"(y)) < ô. 


A discrete time dynamical system (X, F) is equicontinuous if and only 
if it is equicontinuous at every z € X. 


Sensitivity and equicontinuity are related, by comparing the definitions 
we can easily see that 


F is not sensitive = dx: F is equicontinuous at x. (5) 


For linear CA we have that (€? , F) is equicontinuous if and only if it is 
equicontinuous at a single point x € ER . So (5) becomes 


(P, F) is not sensitive & (P, F) is equicontinuous. (6) 


4 Computing Entropy - Statement and Proof of 
Main Theorems 


4.1 Theorem 1 


Let (€1, F) be a 1-dimensional CA over Zm with local rule f(z ,,...,2,) = 
D; Ogi, and let m = př ees pen be the prime factor decomposition of 
m. For i = 1,...,h define 


P; = {0} U {j : ged(asp) = lh Li = min P;, R; = max 2. 
Then, the right and left Lyapunov exponents of (., F) are 
NNNM M" 
AD = min {Li} and A moas (8i (7) 


4.2 Proof of Theorem 1 


Let (P, f) be a linear CA, and let q be any factor of m. For any configu- 
ration c € GP, [c], will denote the configuration in CP defined by 


[cla(v) = [c(v)]q = e(v) mod q for all v € ZP. 
Similarly, F} will denote the map [F], : EP — GP defined by [F]q(c) = 
[F (e)la: 
We will need the following lemma, whose proof is provided in [3]. 


Let (Ce F) be a linear 1-dimensional CA over Z, (p prime) 
with local rule f(z1,...,25) = [X 7; aivi],s. Assume there ex- 
ists a, such that gcd(a;, p) = 1, and let 


P = (j : ged(aj, p) = 1}, L=minP, R = max P, 


Then, there exists h > 1 such that the local rule f) associated 
to F^ has the form 


hÊ 
PE iust) = » bir;| | with gcd(b,;,p) = gcd(b,g, p) = 1. 


i—hL pk 
(8) 


The proof goes by way of taking the power series of the local rule and then 
uses induction to show that f^ has the correct form. 


The proof will be given only for the left Lyapunov exponent A^ since 
the proof for AT is analogous. We know that, since F is a linear map, Lya- 
punov exponents are independent of the particular configuration considered. 
Hence, in the rest of the proof we can safely write A^ and A^ istead of A^ (x) 
and A; (x). 

We first consider the case m = p^ with p prime. From the lemma we 
know that there exist h > 1 and R € Z such that f is permutative in the 
variable x „g and does not depend on any other variable x; with j > hR. 
Let Aj, denote the left Lyapunov exponent of the map F ^ IfR «0 we 
have that f? does not depend on variables with positive index. Hence, 
if two configurations differ in cells with index i > i such differences never 
propagate left under iteration of F. We conclude that Aj, = 0. Assume 
now that R > 0. Let x and 2’ be two configurations such that «(i) = z’ (i) 
for every i < i and x(1) 4 a' (1). Since d (h) is rightmost permutative, we have 
[F (£) (i) = [FP (x ")](i) for every i < i—hR and z(i — AR) # 2/(i—hR), i 
the difference in cell i moves left exactly hR positions. Hence Apn = ES 
We now show that Ar, = hÊ implies Az p — R. From (4) we have 


Since R — max(0, R) then Ap — R and the proof is complete. 

Consider now the general case m — pq, where gcd(p,q) — 1. By the 
Chinese Remainder Theorem we know that the ring Zm is isomorphic to the 
direct product Zp ®Z,. Hence, F” can be expressed as a linear combination 
of |F]; and |F]7 as follows 


F” = ag[F]5 + Bp|F]; 


where [oq]; = 1 and [8p]; = 1. This means that two configurations F"(z) 


and F'"(y) differ in the ith cell if and only if the configurations [F]5 (|z]p) 


and [F]5([y]p) or the configurations [F]? ([z]p) or [F]; ([y]p) differ in the ith 
cell. Hence, differences in cells with index i > i can propagate left to the 
farthest cell reachable by either [F]; or [F];. Hence, A; = max(Arg Am) 
'The proof of the theorem follows by a simple inductive argument on the 


numb of primes in the factorization of the modulus m. 
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4.3 Theorem 2 


Let (1, F) be a 1-dimensional CA over Zm with local rule f(z ,,...,2,) = 
D; ans, and let m = př ees p^ denote the prime factor decomposi- 
tion of m. Let L; and R; be defined as in Theorem 1. Then we can express 
the topological entropy, #, in the closed form, 


h 
H (Ch F) = X ki(Ri — Li) log(p:). (9) 


i=1 


4.4 Comments on Theorem 2 
Theorem 2 follows from a lemma proved by D’amico et al. in [3]. 


Letf(r.,,...,2,) = [Xir tizi] be any linear local rule 
defined over Zx with p prime. Let F be the 1-dimensional global 
transition map associated to f. Let 


P= {0}U {j : ged(aj,p) 21), L= min P, and R = max P. 


Then 
AF (6, F) = KR — L) log(p). 


'This lemma is proven using theorem 1 and applying the definition of topo- 
logical entropy given in [1]. 


4.5 Putting it all together 


Using the previous theorems the authors go on to show that a D-dimensional 
linear CA over Zm with D > 2 has F sensitive to initial conditions and 
entropy that is infinite or F is equicontinuous and the entropy is zero. This 
gives a complete characterization of linear CA over Zm. These results are the 
first step in understanding which classes of CA have Lyapunov exponents 
and topological entropy that are computable. The next step is to attempt 
to generalize to general CA, not just linear. 


5 Further Applications 


The more we understand the computability and entropy of different cellular 
automata the better we can create models which accurately predict the 
future. This paper is huge step towards solving problems 1 and 2 presented 
in [8]. 
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Problem 1 is about finding an overall classification of cellular automaton 
behavior. This approaches that by quantifying the entropy in any linear 
system. We know that if the entropy is infinite that the system will not yield 
a homogeneous state and if it is zero that it either evolves to a homogeneous 
state or simple separated periodic structures. Completing this classification 
would allow us to further understand cellular automata and the systems 
which they model. 

Problem 2 is about the relationship between entropies and Lyapunov 
exponents. This paper solves that problem for linear CA in Zm. This bridges 
the gap where, intuitively the rate of divergence of trajectories and the 
entropy of the system ought to be related, but it was previously unproven. 
This provides hope that we will be able to find a closed form relating the 
Lyapunov exponents and entropies for different types of CA. 

Understanding and being able to compute entropies for cellular automata 
may lead to the development of CA that can model thermodynamic systems. 
It is thought that CA may be the key to understanding a large number of 
subjects on a deeper level. Some wonder if the current model of physics 
(physics with particle-like objects) could be a cellular automata at its most 
fundamental level [2]. 
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